Handling Inconsistency for Multi-Source Integration

The overwhelming amount of information sources now available through the internet has increased the need to combine or integrate the data retrieved from these sources in an intelligent and efficient manner. A desirable approach for information integration would be to have a single interface, like the SIMS information broker [1], which allows access to multiple information sources. An example application is to retrieve all the menus of restaurants from Joe’s Favorite Restaurants site which have been rated highly by the Department of Health. This task, if performed manually, would require a significant amount of work for the user. An information broker, like SIMS, would allow access to multiple information sources, abstracting away the need for the user to know the location or query access methods of any particular source. SIMS stores knowledge about the data contained in each of these sources, as well as the relationships between the sources in the form of a model, called a domain model. The first step in creating the domain model is to determine which data instances appear in multiple sources, e.g. which restaurants from Joe’s web site, like “Art’s Deli,” also appears on the Health Department’s site. Once the common data instances are determined, the relationship between the data in the sources can be modeled in the domain model as subset, superset, equality, or overlapping A special case for information integration is when data instances can exist in inconsistent formats across several sources, e.g. the restaurant “Art’s Deli” can appear as “Art’s Delicatessen” in another source. For the integration process each source can be seen as a relation; therefore, integrating the sources requires performing a join on the two relations by comparing the instances of the primary keys. Since the instances have inconsistent formats, some mapping information is needed to map one instance to another e.g. (“Art’s Deli” from Joe’s source to “Art’s Delicatessen” from the Department of Health site). This information can be stored in the form of a mapping table, or as a mapping function, if a compact translation can be found to accurately convert data instances from one source into another. Once a mapping construct is created it can be modeled as a new information source. This Copyright © 1998, American Association for Artificial Intelligence (www.aaai.org). All rights reserved. integration technique allows SIMS to properly integrate data across several sources that contain inconsistent data instances. Presently, mapping constructs are generated manually, but we are developing a semi-automate approach. The figure contains tuples from Joe’s Restaurant source and the matching tuples from the Health Department: